Review

Accepted Sep 10, 2019; E-published Sep 10, 2019

Acta Derm Venereol 2020; 100: XX–XX.

Corr: Helen Elizabeth Smith, Lee Kong Chian School of Medicine, Nanyang Technological University, 11 Mandalay Road, Singapore 308232. E-mail: h.e.smith@ntu.edu.sg

Assessment of the effectiveness of teledermatology is challenging due to different outcome measurements utilized. This review mapped 55 different outcome measurements reported in clinical trials of teledermatology using the Core Outcome Measures in Effectiveness Trials taxonomy. Each trial used a mean of 3.7 measurements (range 2–7), and most measurements measured “skin and subcutaneous tissue outcomes”. The most frequently used measurement was used in only 3 studies. Over 60% of the measurements did not cite evidence of validation. This review provides a list of measurements for use in designing future teledermatology trials, and provides the foundation to develop a core outcome set.

Skin diseases are one of the most common reasons for patients to seek medical consultations (1). It is also recognized that there is a shortage of healthcare professionals with the relevant skills (2). Dermatology, because of its visual character, is well suited to telemedicine for patient consultations, referrals and triage, which has the potential to increase accessibility to dermatological expertise, maximize work-force potential, improve patient health outcomes, and reduce costs (3). Teledermatology consultations can be “store and forward”, with electronic digital images sent to review at a later time (also referred to as asynchronous), live and interactive (synchronous) or a combination of both (3). Literature reviews of teledermatology service evaluations have reported positive impacts, such as more rapid diagnoses (4), improved cost-effectiveness (5, 6), but also some negative impacts, such as increased referrals to secondary care (7). Systematic literature reviews of randomized controlled trials (RCTs) of telemedicine tend to be more reserved about potential benefits because of the heterogeneity in quality, design, conditions and outcomes of the studies, which in turn limits the ability to pool data (8–10).

The lack of standardization of outcome measurement instruments is a recurrent challenge when making evidence-based decisions to optimize patient care. However, this problem tends to persist, because within a tight project timeline, researchers may not have the resources to assess the range of outcome measurement instruments used previously, or to identify those that would enable direct comparisons with previous work. To address this issue the Core Outcomes Measures in Effectiveness Trials (COMET) Initiative (http://www.comet-initiative.org/) is now encouraging researchers to develop and adopt the use of evidence-based core outcome sets (COS) (11). These are agreed standardized sets of outcomes that the COMET Initiative recommends to be measured and reported as a minimum in all clinical trials in a particular condition or context (12). They may also be used in audit or other forms of research. A taxonomy to classify outcomes has also been developed by the COMET Initiative, to standardize the classification of all outcomes reported. This taxonomy is also used in the classification of outcomes in COS, which further encourages the standardized reporting of outcomes (11). One important step in the development of a COS for a particular field is to identify outcome measurement instruments used previously in order to generate a long list of outcomes that can be considered candidates for inclusion into a particular COS (13) This is typically followed by some form of consensus-seeking process (such as an e-Delphi followed by a consensus meeting of all interested stakeholders) with the ultimate goal of agreeing on a COS (11). This study has been designed to identify and categorize the outcome measurement instruments reported in RCTs of teledermatology interventions.

Inclusion and exclusion criteria

This systematic mapping review protocol defined study inclusion criteria as RCTs, cluster randomized controlled or quasi-randomized trials of teledermatology interventions in which participants were patients presenting with dermatological problems. The study findings had to be published as peer-reviewed, full-text articles within the last 10 years. Studies with teledermatology services as an intervention and standard care as the control group were included. Articles were not limited to the English language or to any particular age group.

Systematic reviews, editorials, commentaries or letters were excluded. Similarly, articles were also excluded if they focused on the evaluation of a technology or a device without patient involvement, or if the intervention used was not teledermatology; for example, outreach consultant care or general practitioners (GPs) with a special interest in dermatology.

Search strategy

The search strategy was developed with the medical librarians at the Lee Kong Chian School of Medicine (Table SI) and conducted in November 2018. MEDLINE, EMBASE, CINAHL, PubMed, and Scopus were searched for articles published between 1 January 2008 and 31 December 2018. The search was complemented by hand-searching of trial registries (e.g. Clinicaltrials.gov), targeted journals (e.g. Journal of Telemedicine and Telecare, Telemedicine Journal and e-Health), the Cochrane Controlled Register of Trials, and the reference lists of all eligible studies.

Eligibility assessment

Two reviewers (AC and CS) independently screened the titles and abstracts for eligibility based on the above selection criteria. Where consensus of eligibility was not reached a third reviewer, (HES or ChA) was consulted. Full texts were obtained for all selected studies, and if study eligibility remained unclear it was again discussed with a third reviewer.

Data extraction

Characteristics of the studies (i.e. year published, study setting, country, skin disease studied, age and sex of participants), outcome reported, type of outcome reported (i.e. primary or secondary), outcome measurement instrument used, and the remarks about the validity of the outcome measurement instrument made by the studies authors’ were extracted from the eligible papers. Outcomes were mapped onto the taxonomy developed by the COMET initiative (12). If an outcome was composite and addressed several domains it was classified within each of the relevant domains.

Studies and study characteristics

After duplicates were removed, 460 potentially eligible records were identified and screened according to the protocol (Fig. 1). A final total of 16 articles based on 12 studies were included in this review. Data were extracted from all the articles, with one exception, an article in Dutch (14) that reported the same results from a study that had been published previously in English (15). Most of the studies included in this mapping review were conducted in the USA (64.3%) and the rest in Europe (i.e. Austria, France, Norway, Switzerland, and The Netherlands). The study characteristics of the studies were as follows: a total of 2,993 participants were recruited (ranging from 64 to 698 participants per study). The mean age of participants ranged from 2.7 to 63 years (but only 9 studies reported this). In the 10 studies that reported the sex of participants, slightly more men (54.3%) were included than women (45.7%). Full details are shown in Table I.

Outcome measurement instruments

The total number of outcome measurement instruments used was 55, with a mean of 3.7 in each article (range 2–7). Twenty-four of the outcome measurement instruments were categorized in the Life Impact COMET Core Area, with 2 of these outcome measurement instruments also categorized in the Resource Use COMET Core Area. Seventeen outcome measurement instruments were categorized in the Physiological/clinical COMET Core Area, and 11 were in the Resource Use COMET Core Area. The heterogeneity of outcome measurement instruments identified in this review is further exemplified by the fact that the Dermatology Life Quality Index (DLQI), which was the most commonly cited outcome measurement instrument, represented only 3 (5.5%) out of 55 outcome measurement instruments. These details are further detailed in Table II.

Outcomes and outcome domains

The total number of primary and secondary outcomes reported in the 16 articles was 44; hence the mean number of outcomes reported was 2.9. Slightly over half (53.3%) of the articles differentiated between primary and secondary outcomes. From these studies, 11 primary outcomes and 23 secondary outcomes were reported. A total of 21 outcomes were reported in the studies that did not differentiate between primary or secondary outcomes.

As shown in Fig. 2, when mapped on the COMET taxonomy the “Skin and subcutaneous tissue outcomes” outcome domain had the largest number of outcome measurement instruments mapped in it (i.e. 34.5%). This domain is in the “Physiological/clinical skin” COMET Core Area, which includes physiological symptoms and functioning (12). Despite having the highest frequency, the relatively low percentage in this domain reflects the heterogeneity of the outcome measurement instruments reported in the studies. The most common outcome measurement instrument used that was mapped in this domain was the “Investigator Global Assessment”. The second most commonly mapped COMET outcome domain was “Delivery of Care” (i.e. 21.8%), and this domain is in the “Life Impact” Core Area. Only one of the outcome measurement instruments mapped in this domain was validated; this was the Dutch translation of the Patient Satisfaction Questionnaire III, which was modified and revalidated because only 20 out 43 questionnaire items were used. The third most commonly mapped COMET domain was “Economic” (i.e. 14.5%), and this is in the “Resource Use” COMET Core Area. The most common outcomes in this domain were direct costs and indirect costs.

Validation

Of the 55 outcome measurement instruments, 61.8% did not have citations of validation in the study publication.

Safety outcomes

There were no specific safety outcomes measured in the studies; however, 4 studies briefly mentioned issues about safety and adverse events. One study reported that participants could report any adverse events that occurred on a standardized questionnaire used during the trial (16). Another study, which involved isotretinoin therapy for participants, collected reports of adverse reactions from clinicians during face-to-face or online consultations, depending on which experimental group the participant was allocated into (17). Two other studies mentioned safety only as part of the discussion of their results (18, 19).

To the best of our knowledge, this is the first systematic review of outcomes and outcome measurement instruments reported in teledermatology RCTs. Sixteen articles from 12 eligible studies were included in this review. It was notable that the included studies were either from Europe or from the USA.

Heterogeneity of outcome measurement instruments

There were 44 outcomes reported, and the majority of outcomes were categorized as skin and subcutaneous tissue outcomes. This finding is similar to what was found in another systematic review that identified and grouped outcomes of dermatology-related RCTs (20). Of the 55 outcome measurement instruments used to measure these outcomes, only 3 of these instruments were reported in different articles. This highlights the heterogeneity of outcome measurement instruments used in RCTs of teledermatology, and questions the comparability of these trials. The heterogeneity of outcome measurement instruments found in this review has also been reported in other systematic reviews (21, 22).

Validation of outcome measurement instruments

Over 60% of the reported outcome measurement instruments did not have any citation to a validation study. It was beyond the scope of this review to explore further the rigour of the validation of the outcome measurement instruments. Citing the validation or development references, helps the clinical and scientific community to make informed decisions about the outcome measurement instruments they can use for their own clinical use and research studies.

Safety

There were no specific safety outcomes measured in the reviewed trials, but 4 studies briefly mentioned issues about safety and adverse events. While safety may not be of great importance in studies focusing only on teledermatology referral processes, when the study includes treatment or procedures then safety is increasingly important. The Patient-Reported Outcomes Safety Event Reporting Consortium Guidance could be used to guide such a practice in the future (23).

Strengths and limitations

The results from this mapping review provide novel and valuable information about outcome measurement instruments that clinicians and researchers can use to make informed decisions about which outcome measurement instrument to use for treatment and research studies. Specifically, we have generated a list of the outcome measurement instruments used in recent RCTs of teledermatology and the reported validity of each measure. This information will provide a ready resource of outcome measurement instruments for researchers of teledermatology in the future. These data may also inform the process of developing a core outcome set in the future.

The current review has some limitations. First, the search was limited to trials published in the last decade. While this ensures an up-to-date overview of recent trials, many studies were excluded, as the rate of teledermatology trials conducted was low in the inclusion period of this review. Secondly, the current review excluded unpublished research reports and conference abstracts, in which additional outcome measurement instruments might have been found. Thirdly, an in-depth analysis of the validity of outcome measurement instruments used was not undertaken. The scope of this mapping review was constrained by the resources available, but future reviews could expand the current review to address the second and third limitations.